Sql extended with transient fields for calculation expressions in enhanced data models

ABSTRACT

A database language (e.g. SQL) is extended to define a transient field whose value is derived from data stored in a database. The value of the transient field is calculated by the database engine for consumption as needed by the application layer, for example as part of a particular data model. The value of the transient field may not be materialized as a persistent field in the underlying database table, or may be selectively materialized based upon a heuristic or hint. Certain embodiments may implement the transient field as a basic elaboration on standard SQL utilizing an identifier. Some embodiments may implement the transient field as an entity where SQL has been extended to accommodate concepts of a higher-level Entity Relationship Model (ERM). The use of constants and/or floating values (e.g. a current time) in connection with transient fields, is also disclosed.

BACKGROUND

Embodiments relate to databases, and in particular, to extending a database language to accommodate calculation expressions in enhanced data models.

Unless otherwise indicated herein, the approaches described in this section are not prior art to the claims in this application and are not admitted to be prior art by inclusion in this section.

Many database structures rely upon Structured Query Language (SQL) as the standard approach to define, read, and manipulate data within a database. At a low level, such a database may employ fundamental data definition and processing that is based upon a relational model. In particular, a data definition defines a data type with sufficient metadata being associated therewith. A data definition may also involve definition of a database structure such as columns and tables. Many database structures rely upon Structured Query Language (SQL) as the standard database language to define, read, and manipulate data within a database. In its standard form, SQL itself reflects the basic relational model of the database. Various other types of applications (e.g. toolsets) are constructed by developers to allow consumers to interact with the database in an efficient and intuitive manner. Such applications are typically provided in an application layer overlying the database.

The overlying applications, such as consumer technology and toolsets provided by developers, may introduce higher-level models, e.g., entity-relationship models (ERMs) in order to contribute semantics and ease consumption by the user. In particular, a plain data model on the SQL level only contains the requisite information to process data on the SQL-level. Adding more information in a declarative fashion provides potential for higher-level engines to offload work from developers by contributing more semantics. Adding more information in a declarative fashion can also make data models more comprehensible, thereby easing their consumption by users.

One example of a higher-level model is an OData Entity Data Model (EDM). In particular, OData is a web protocol standard providing platform-agnostic interoperability for querying and updating data. OData leverages web technologies such as HTTP, Atom Publishing Protocol (AtomPub), and JSON (JavaScript Object Notation) in order to provide access to information from a variety of applications. The simplicity and extensibility of OData can provide consumers with a predictable interface for querying a variety of data sources.

Other examples of higher level models may include the Semantic Layer in the Business Intelligence (BI) platform of SAP AG, in Walldorf, Germany, Java Persistence API (JPA) and enterprise objects in Java, or the business objects frameworks in Advanced Business Application Programming language (ABAP). Also, the River programming model and the River Design Language (RDL) of the River Application Development framework for SAP AG, in Walldorf, Germany, are based upon entities linked by relationships.

Even though those higher-level models may share many commonalities, the individual information cannot be shared across stacks. That is, the higher-level models mentioned above contribute essentially the same kind of additional information, yet that information is provided in different ways that interfere with its being shared across higher level models (e.g., between an OData EDM and an ERM created using RDL).

This situation results in a fragmented environment, with information unable to be shared between applications. To cope with this fragmentation, redundant information is provided with application developers and customers contributing the same essential information in multiple forms, thereby undesirably increasing overhead.

Furthermore, while the developers of consumer technologies may have some knowledge of SQL, they are generally not experts in complex SQL programming.

It is also noted that when retrieving data from tables in a database, new data attributes may be calculated transiently based on the values of other attributes, utilizing a calculation expression. In order to interact with a relational database utilizing conventional SQL, such calculation expressions may need to be redundantly placed in many locations within the code. Thus, there is a need for an improved language for interacting with relational databases.

SUMMARY

A database language (e.g. SQL) is extended to define a transient field whose value is derived from data stored in a database. The value of the transient field is calculated by the database engine for consumption as needed by the application layer, for example as part of a particular data model. The value of the transient field may not be materialized as a persistent field in the underlying database table, or may be selectively materialized based upon a heuristic or hint. Certain embodiments may implement the transient field as a basic elaboration on standard SQL utilizing an identifier. Some embodiments may implement the transient field as an entity where SQL has been extended to accommodate concepts of a higher-level Entity Relationship Model (ERM). The use of constants and/or floating values (e.g. a current time) in connection with transient fields, is also disclosed.

An embodiment of a computer-implemented method comprises providing a database comprising data of a first type and data of a second type, and causing a database engine to receive from a query engine, a query in a database language defining a transient field. The database engine is caused to communicate with the database to generate a query result from the query, wherein the database engine fills in a value for the transient field derived at least in part from the data of the second type. The database engine is caused to communicate the query result to the query engine for display to a user.

An embodiment of a non-transitory computer readable storage medium embodies a computer program for performing a method comprising providing a database comprising data of a first type and data of a second type, and causing a database engine to receive from a query engine, a query in a database language defining a transient field. The database engine is caused to communicate with the database to generate a query result from the query, wherein the database engine fills in a value for the transient field derived at least in part from the data of the second type. The database engine is caused to communicate the query result to the query engine for display to a user.

An embodiment of a computer system comprises one or more processors and a software program executable on said computer system. The software program is configured to provide a database comprising data of a first type and data of a second type, and cause a database engine to receive from a query engine, a query in a database language defining a transient field. The database engine is caused to communicate with the database to generate a query result from the query, wherein the database engine fills in a value for the transient field derived at least in part from the data of the second type. The database engine is caused to communicate the query result to the query engine for display to a user.

In an embodiment, the database language comprises SQL.

According to some embodiments, the transient field is defined by an identifier in the database language.

In various embodiments, the database language is extended to include an entity, and the transient field is defined by the entity.

In certain embodiments, the value is not materialized in a persistent field in the database.

According to some embodiments, the value is materialized in a persistent field of the database based upon a heuristic or a hint.

The following detailed description and accompanying drawings provide a better understanding of the nature and advantages of the present invention.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 shows a simplified view of a database system according to an embodiment;

FIG. 2 shows an enlarged view of the database structure of FIG. 1;

FIG. 3 illustrates relationships between individual languages making up a language family useful for interacting with a database;

FIG. 4 is a simplified view showing an approach for extending SQL according to embodiments;

FIG. 5 is a simplified diagram illustrating a process flow according to an embodiment;

FIG. 6 illustrates hardware of a special purpose computing machine configured to implement transient fields according to an embodiment;

FIG. 7 illustrates an example of a computer system;

FIG. 8 shows an embodiment of a database system according to an embodiment; and

FIG. 9 shows an embodiment of a simplified process flow according to an embodiment.

DETAILED DESCRIPTION

Described herein are techniques for extending a database language to accommodate transient fields for calculation expressions in enhanced data models. In the following description, for purposes of explanation, numerous examples and specific details are set forth in order to provide a thorough understanding of the present invention. It will be evident, however, to one skilled in the art that the present invention as defined by the claims may include some or all of the features in these examples alone or in combination with other features described below, and may further include modifications and equivalents of the features and concepts described herein.

According to embodiments, a database language may be extended to include a transient field whose value is derived from data stored in a database. The value of the transient field is not stored in an underlying database table. Instead, the value of the transient field is calculated by a database engine for consumption as needed by an application layer, for example in a calculation expression of a particular data model.

Conventionally, when retrieving data from tables in a database, new data attributes may be calculated transiently based on the values of other attributes. An example of this is given below:

SELECT name, years(now—birthday) as age FROM Employee

However, there is a need to redundantly place the above expression in many different locations, i.e., wherever the definition of “Employee” is consumed.

In order to address this issue, embodiments introduce the concept of pre-defined calculated transient fields in data models. According to certain embodiments, such pre-defined calculated transient fields are implemented based on a data definition language (DDL). The DDL is used for defining semantically rich data models, including the data types, associated metadata, and database organization (e.g., columns and tables). According to some embodiments, the DDL may be extended to further enrich these data models through the use of entities and annotations.

FIG. 8 shows a simplified view of an embodiment. In particular, FIG. 8 shows system 800 comprising a database 802 located within a database layer 804. The data within the database may be organized as a table 806 comprising similar types of information grouped into columns 808. A database engine 810 is also located in the database layer 804. The database engine 810 operates to control the flow of data to and from the database 802. In particular, the database engine 810 controls the flow of data between the database 802 and an overlying application layer 820.

The application layer 820 comprises a query engine 822 that is configured to communicate with the database engine 810. The application layer 820 further includes a data model 824 that organizes data of the underlying database 802, in ways that are useful to an end user 830. In this highly simplified example, the data model 824 may include a value that represents a combination of certain types of information present in the underlying database 802.

For example, the data model may include a value (Z), that represents the sum of different types of information (A, F) taken from the database. As disclosed herein, one of those types of information (F) may actually represent the combination of other pieces of information stored in the database (e.g., the difference between two values D and E actually stored in separate columns of the underlying database table).

In one conventional approach, this quantity (D-E) could be calculated in advance for each row of the table, with those values stored in a separate column within the database for later access if/when it is consumed by the application layer. Such a conventional approach, however, may require high memory consumption to store all of the combinations.

In another conventional approach, the application layer could include a separate calculation expression each time the quantity F (D-E) is consumed by the data model. As mentioned above, however, such a conventional approach involves redundant instances of the same calculation expression within the programming code of the application.

In contrast with such conventional approaches, embodiments extend the database language to include a transient field. In certain embodiments, the value of this transient field is calculated by the database engine 810 and populated during query execution time when consumed by the application layer 820. The calculated value of the transient field may then be communicated from the database engine 810 to the query engine 822, for display to the end user 830 as part of the data model 824 of the application layer 820. In the highly simplified view shown in FIG. 8, the transient field F is part of a query 850 in a database language defining the transient field; its value is determined as the difference of D-E in the database 802. The value of Z is returned to the query engine as a query result 852. In particular, the transient field is incorporated into calculation of the component Z of the data model 824 as the sum of A and F.

According to some embodiments, the value of the transient field is not materialized in a persistent field in the underlying database table. This non-materialization of the transient field is depicted by “- -” in FIG. 8.

In other embodiments, value of the transient field may be materialized in a persistent field in the underlying database table 806. Such materialization of the transient field value is depicted by “#” in FIG. 8. The materialization may be based upon a heuristic or hint 890.

It is noted that the value of the transient field need not be determined exclusively from values stored in the underlying database 802. In certain embodiments, the value of the transient field could be determined in part on the basis of a constant. Also, in certain embodiments the value of the transient field could be determined in part utilizing a floating value, such as for example, a present time.

FIG. 9 is a simplified view of a process flow 900, according to an embodiment. In a first step 902, a database is provided comprising data of a first type and data of a second type. In a second step 904, the database engine is caused to receive from a query engine, a query in a database language defining a transient field.

In a third step 906, the database engine is caused to communicate with the database to generate a query result from the query. The database engine populates a value for the transient field derived at least in part from the data of the second type.

In a fourth step 908, the database engine is caused to communicate the query result to the query engine for display to a user.

In order to provide additional understanding regarding various embodiments, several examples are now provided below in connection with SQL as the database language. The first two examples illustrate the use of the transient field “grossamount”, in an application layer data model comprising a table including an “amount” and a “taxrate”. The first example shows an embodiment implementing the transient field as a basic elaboration upon standard SQL. The second example shows an embodiment implementing the transient field as an elaboration upon SQL that has been extended to accommodate concepts of a higher-level Entity Relationship Model (ERM).

Example 1

In this first example, the transient field “grossamount” is implemented directly in SQL in order to create the underlying table Foo consumed by the application layer. This transient field is defined in the SQL by an identifier (“TRANSIENT”). This transient field is determined from the values “amount” and “taxrate” present in the underlying database table, but is not stored (i.e. as indicated below with the shorthand “- - - - -”) in that underlying database table.

CREATE TABLE foo { amount, taxrate, TRANSIENT grossamount AS amount * ( 100% + taxrate ) }

TABLE Foo amount taxrate _grossamount _(—) e1 100 19% — 200 20% —

Example 2

In this example the transient field “grossamount” is implemented as an extension of SQL that accommodates aspects of a higher-level Entity Relationship Model (ERM). Such an extension is described in detail below in connection with FIGS. 1-7.

According to specific embodiments, a DDL such as in SQL may be extended to introduce the “=” expression in order to create a pre-defined calculated transient field. In particular, the syntax of element definitions is enhanced to allow specifying calculated fields as follows:

ElementDefinition += | CalculatedField CalculatedField = [ element ] elementName [ “:” AssignedType ] “=” Expression “;”

The value of such a calculated field is calculated through the given expression at runtime. Such fields are read-only in a sense that no other values can be written to them.

The following corresponds to implementation of a transient field utilizing SQL extended to accommodate features of an entity-relationship model. In particular, this second example shows definition of an entity foo which includes the transient field “grossAmount”.

type Taxrate : Decimal(3,2) default 19%; type Payslip { amount : Decimal(10,2); taxrate : Taxrate; grossAmount : type of amount = amount * (100% + taxrate); kind : String(7) enum { regular=‘regular’; bonus=‘bonus’; } payoutDate : Date; } entity foo { element e1 : ... element payslip: Payslip { amount taxrate grossamount ... } element newGrossAmount : type of amount = amount * 200% }

Example 3

Some implementations for calculated transient fields, may simply expand them in queries with the calculation expression. For example, given the following definition of a calculated field age:

entity Person { ... birthday : Date; age = years(now−birthday); } the following query:

SELECT age from Person;

would be expanded to an expression resembling the following:

SELECT years(now—birthday) as age from Person;

In this example, additional information beyond that specifically stored in an underlying database, is relied upon. In particular, the floating value “now” specifies the present time, a quantity which is not specifically stored in the underlying database table.

Users can rely on this understanding of the behavior of a calculated field, as if they were transient fields populated during query execution time. Yet, various implementations may internally map this behavior to more appropriate strategies. One example is having the calculated transient values actually be materialized in persistent fields in the underlying tables, based on heuristics or hints.

In one particular example, ‘hints’ would be directives added by the developer to the field's declaration in the data model. Such a ‘hint’ could be used to determine an implementation to materialize a calculated field as a persistent field.

An overview of DDL syntax in SQL is provided below. The following shows top-level syntax elements of the DDL in SQL extended according to an embodiment as described herein.

DataModelDefinition = [ NameSpaceDirective ] UsingDirective* DataTypeDefinition* DataTypeDefinition = TypeDefinition | EntityDefinition | ViewDefinition - - abbreviated EntityDefinition = [ define ] entity entityName StructuredType [ “;” ] - - abbreviated ViewDefinition  = [ define ] view entityName as QL:query [ “;” ] - - abbreviated TypeDefinition = type typeName TypeSpec TypeSpec = “:” AssignedType “;” | StructuredType [ “;” ] StructuredType = “{” ElementDefinition* “}” ElementDefinition = [ element ] elementName TypeSpec AssignedType = DerivedType | InferredType DerivedType = existingType [ “(” paramValue ( “,” paramValue )* “)” ] InferredType = type of [ existingType | elementName ] ( “.” elementName )* typeName, existingType = QualifiedName elementName = Identifier paramValue = Literal

Certain embodiments may allow for the inference of type. Specifically, under certain circumstances explicit type declarations, for example [“:” AssignedType], may be omitted for constants and calculated fields. This may be done if, and only if, a type (including relevant details such as dimension of a string; precision/scale of a decimal) can unambiguously be inferred from a given expression.

Such an inference would likely not be available for string types used in entity definitions because the maximum length parameter for the string could not be inferred.

Certain embodiments may also allow constants to be defined. In particular, constants can be defined using the same syntax as for calculated transient fields, prepended with the modifier [const] as expressed in the following syntax enhancement:

ElementDefinition += | Constant Constant = const CalculatedField

The modifier “const” turns an element definition into a declaration of an alias name for a literal expression specified via “=” Expression. The element name can be used later on in a symbolic, compiler-checked way wherever a literal value can be put in DDL, Query Language (QL) and Expression Language (EL).

It is noted that constants do not show up in persistence or in runtime structures as they are merely symbols used by the compiler. No value can be assigned at runtime.

SQL Extended to Accommodate ERMs

Described herein are techniques for extending a relational model-based database language (e.g., SQL), to accommodate higher level entity-relationship models. In the following description, for purposes of explanation, numerous examples and specific details are set forth in order to provide a thorough understanding of the present invention. It will be evident, however, to one skilled in the art that the present invention as defined by the claims may include some or all of the features in these examples alone or in combination with other features described below, and may further include modifications and equivalents of the features and concepts described herein.

FIG. 1 shows a simplified view of a database system 100, according to an embodiment. In particular, the database system 100 comprises data 105 of the database itself, organized according to a relational model.

A lower layer 106 of the database system 100 comprises calculation logic 108 that is designed to interact with the data 105 itself. Such calculation logic 108 may be performed by various engines (e.g., SQL engine, calculation engine, SQL script) in order to provide basic data definition and processing based on the relational model. Such basic data definition can include defining of data types making up the database, associated metadata, and the database structure (e.g., columns, tables). The lower layer 106 of the database system may include SQL script 110, as well as data structures such as tables 112, views 114, and calculation views 116.

The embodiment presented in FIG. 1 shows HANA, the in-memory database available from SAP AG of Walldorf, Germany, implemented as the database. However, embodiments are not limited to use with this particular database. Examples of other in-memory databases include, but are not limited, to the SYBASE IQ database also available from SAP AG; the Microsoft Embedded SQL for C (ESQL/C) database available from Microsoft Corp. of Redmond, Wash.; the Exalytics In-Memory database available from Oracle Corp. of Redwood Shores, Calif., etc.

Further, while the embodiment presented in FIG. 1 shows the database as comprising an in-memory database, various embodiments could be employed in conjunction with conventional disk-based database systems.

An application layer 118, overlying the calculation logic 108 of the database system 100, comprises control flow logic 120. The control flow logic 120 may be implemented utilizing River Definition Language (RDL) 122 and JavaScript (JS) 124 to reference model concepts such as entities and relationships that are not reflected in basic SQL. This control flow logic 120 may further comprise common languages for defining and consuming data across different containers (e.g., native, ABAP, Java).

As shown in FIG. 1, in order to facilitate the sharing of information across such different containers and thereby promote a more unified environment, the database system, according to embodiments, further comprises Core Data Services (CDS) component 130. CDS component 130 comprises a common set of domain-specific languages (DSL) and services. The CDS component 130 allows defining and consuming semantically rich data models as an integral part of the database structure. This permits data modeling as well as the retrieval and processing of data to be raised to a higher semantic level that is closer to the conceptual thinking of domain experts. The role of the CDS component 130 is discussed in detail further below.

FIG. 1 further shows client 150 in communication with the HANA in-memory database appliance available from SAP AG. The client 150 includes presentation logic 152 to provide an output 154 comprising data of the underlying database structure, in a form desired by a user. Here, the output is shown as a vertical bar chart, but of course this represents only one of a multitude of different ways in which database data may be communicated to a user. The presentation logic may communicate such output in the form of HTML 156, cascading style sheets (CSS) 158, and/or JavaScript 160, or a variety of other user interface technologies.

FIG. 2 shows an enlarged view of the HANA in-memory database structure of FIG. 1. In particular, FIG. 2 shows SQL engine 200, calculation engine 202, and SQL script 204, as part of the lower layer 106 that performs basic data definition and processing based upon the relational model, according to which the data 105 of the database is organized. FIG. 2 also shows the application layer 118 of the database structure of FIG. 1, including the RDL and JS elements of a query engine 119. The application layer 118 further comprises application containers and other host languages 220, including ABAP 222, Java 224, and others 226.

FIG. 2 further shows the CDS component 130 as lying between the lower layer and the application layer. As illustrated in this figure, the CDS component 130 can be leveraged in any consuming stack variant (stack of software layers located on top of each other), as implemented through the application layer 118. Specifically, services in higher layers can use/consume the services of lower layers. Here, because the application layer sits on top of a data layer in which the CDS component 130 resides, definition and consumption of the semantically rich higher level models is allowed.

In particular, the CDS component 130 implements higher level Domain Specific Languages (DSLs) and services based on an entity-relationship model (ERM). The Data Definition Language (DDL) 230 is used for defining semantically rich data models, including the data types, associated metadata, and database organization (e.g., columns and tables). As mentioned throughout, according to embodiments, the DDL may be extended to further enrich these data models through the use of entities and annotations.

The Query Language (QL) 232 is used to conveniently and efficiently reading data based on data models. It is also used to define views within data models. The role of the QL and its relation to the DDL is further illustrated in connection with FIG. 3.

The Expression Language (EL) 234 is used to specify calculated fields, default values, constraints, etc. within queries. These calculated fields, default values, and constraints may be specified as well as for elements in data models.

Other elements of the CDS component 130 can include Data Manipulation Language (DML) 236 and a Data Control Language (DCL) 237 that helps in controlling access to data.

Embodiments as described herein may distinguish between the domain-specific languages DDL, QL, and EL as members of a language family. This approach fosters considerations such as modular design, incremental implementation, and reuse. FIG. 3 is a simplified view illustrating relationships between these language family members.

A consistent language experience across the members of the family of FIG. 3, can be achieved by ensuring the languages follow a common style. This can extend to the host programming language, with expressions showing up in DDL, QL, and EL code, adopting the same syntax.

Utilization of application level domain language(s) as has been described above, can offer certain benefits. One possible benefit is that the application domain level language can avoid the use of “inefficient” and error-prone code.

Take, for example, the following simple data model describing employee information:

entity Employee { name : String(77); salary : Amount; // a structured type orgunit : Association to OrgUnit; addresses : Association to Address[0..*] via entity Employee2Address; homeAddress = addresses[kind=home]; // introduced later on } entity OrgUnit { name : String(111); costcenter : String(44); manager: Association to Employee; parent: Association to OrgUnit; } entity Address { key streetAddress; key zipCode; city; // omitted type defs kind : enum { home; business; } }

Under some circumstances, it may be desired to write a query statement as follows:

SELECT id, name, homeAddress.zipCode FROM Employee WHERE . . . .

Within that sample snippet, path expressions along relationships are used to fetch data from an associated entity. In the simple data model above, the above query statement is equivalent to the following standard SQL statement:

SELECT e.id, e.name, a.zipCode FROM Employee e LEFT OUTER JOIN Employee2Address e2a ON e2a.employee = e.id LEFT OUTER JOIN Address a ON e2a.address = a.id AND a.type=’homeAddr’ WHERE ...

This statement, however, may already be too complex for many application developers. Thus, code patterns similar to that given below, may be used in some pseudo languages:

customers = SELECT * FROM Customer foreach c in customers do write c.id write c.name addresses = SELECT * FROM Address a, $Customer2Address c2a  WHERE a.id = c2a.address AND c2a.customer = :c.id foreach a in addresses do if a.type = ‘homeAddr’ then write a.zipCode end end

There are several issues with the code presented immediately above. One issue is the use of an imperative coding style with loops in loops, resulting in 1+n queries being executed or too much data being fetched with SELECT * statement.

The above code represents only a relatively simple case. A more complex case is found in the following example:

SELECT FROM OrgUnit[boardarea=‘TIP’] .employees[salary>‘$100,000’] { addresses[kind=home].city, count(*) }

The preceding cases illustrate the importance of increasing expressiveness of the languages used in application development (here, the query language). This allows the intent of application developers to be captured, rather than being buried under substantial volumes of imperative boilerplate coding.

Such expressiveness is in turn is fundamental to having optimizations applied by the query engine (in a manner analogous to functional programming vs. imperative programming). This can affect system characteristics, such as its overall performance and scalability. Further, a language's ability to allow developers to draft concise and comprehensive code can increase developer productivity. It can also reduce the risk of mistakes and also enhance readability, and thus maintainability of the code.

In order to write concise and readable query statements, it is desirable to enrich the data definitions with sufficient metadata (e.g., about associations, semantic types, etc.) Accordingly, embodiments seek to extend the DDL to define that information, and seek to extend the QL to leverage such definitions.

DDL and QL are declarative, domain-specific languages providing developers with concise ways to express their models and queries. Certain concepts may originate from entity-relationship modeling (ERM). By adding native support for such concepts in the underlying engine of the database, embodiments avoid the impedance mismatch induced by the translation of conceptual models based on ERM, into implementations based upon a plain relational model. In particular, writing concise and comprehensive code reduces risks of mistakes and increases readability and maintainability.

Moreover, as the concepts of entity-relationship models may lie at the core of many higher-level models, embodiments are able to capture the semantics of data models created (e.g. in RDL), and share those semantics with database modelers, and/or ABAP or Java consumers. This reduces fragmentation and the loss of semantics.

In addition, since ERM is also the chosen basis for technologies like OData EDM, embodiments can facilitate mapping entities and views to OData entity sets.

Embodiments may employ a functional approach that is based on standard SQL. In particular, the comprehensive, domain-specific nature of DDL and QL allows capturing the intent of application developers, avoiding a lack of clarity regarding that intent which can result from large volumes of imperative boilerplate coding. This follows the principles of functional programming and may be important for optimizations.

The functional approach may be inherited from SQL. A SQL SELECT statement declares which subset of an overall data model is of interest as projections and selections. It may be left to the query engine to determine optimal execution, including parallelizing as appropriate.

In contrast with imperative object traversion patterns, embodiments can speed up many data retrieval use cases. While many of those retrieval cases are not individually expensive, the cumulative impact of this streamlining can have significant impacts on scalability, as it affects all requests over a long period of time.

Embodiments address some of the complexity offered by standard SQL to typical application developers by raising the basis of SQL from plain relational models to the level of conceptual models. This is done by providing native support for ERM in the database system. In this manner, the use of SQL may be reestablished for most application developers, not only for those with the SQL expertise for specific optimization tasks.

Embodiments employ associations in DDL. Specifically, the DDL allows definition of data models as entity-relationship models on a semantically rich level that is close to actual conceptual thought. To achieve this over the conventional relational model of standard SQL, certain concepts are added.

FIG. 4 is a simplified view showing an approach for extending SQL according to embodiments. As shown in the system 400 of FIG. 4, one concept underlying embodiments as described herein, is the use of entities 401 with structured types. This is in contrast with a conventional relational database, which uses only flat tables.

Entities are structured types with an underlying persistency and a unique key 402. Structured types are records of named and typed elements. An entity key is formed of a sub-set of the elements of the entity that uniquely identify instances. Views are entities defined by a query, which essentially defines a projection on underlying entities.

Another concept underlying entities as described herein, involves employing associations 404 on a conceptual level. This approach contrasts with the conventional use of hand-managed foreign keys.

Associations define relationships between entities. They are specified by adding an element with an association type, to a source entity 408 that points to a target entity 410. As shown in the FIG. 4 the relationship implemented by the association type, between source entity type and the target entity type, reflects the actual relationship between entities in the overlying ERM model 420. Using the type definition, associations may capture metadata about relationships present in the ERM model in a ‘reflectable’ way. According to such a reflectable characteristic, a consuming portion of code receiving a piece of data from the database can get back to the type information (i.e., metadata) provided for the respective elements in the data model.

The association may be complemented by optional further information (e.g., regarding cardinality, which keys to use, additional filter conditions) up to a complete JOIN condition. According to embodiments, the clause-based syntax style of standard SQL may be adopted for specifying the various parameters without sacrificing readability.

In addition, the extended DDL works with Custom-defined Types instead of being limited to primitive types only. The extended DDL may also add some other enhancements, such as annotations to enrich the data models with additional metadata, constraints, or calculated fields.

FIG. 5 is a simplified diagram illustrating a process flow 500 according to an embodiment. In a first step 502, a database is provided comprising data organized according to a relational model.

In a second step 504, a database engine is provided in communication with a database utilizing a language describing the relational model. In a third step 506, an application is provided comprising an entity-relationship model (ERM) including a first entity, a second entity, and a relationship between the first entity and the second entity.

In a fourth step 508, a query engine of the application communicates a query to the database engine utilizing a language extension providing the entity and relationship components of the ERM. The language extension may comprise a first structured entity type including a first key and indicating the first entity, a second structured entity type including a second key and indicating the second entity, and a third structured association type reflecting the relationship. The association type may be complemented with further additional information.

In a fifth step 510, the database engine returns a query result to the query engine based upon the language extension.

EXAMPLES

Some examples of extension of the SQL database language to provide entities and associations of ERMs, are now given below:

entity Address { owner : Association to Employee; // can be used for :m associations streetAddress; zipCode; city; // snipped type defs kind : enum { home, business }; } entity Employee { addresses : Association[0..*] to Address via backlink owner; homeAddress = addresses[kind=home]; // → using XPath-Like filter. } Association to Address; Association to Address { zipCode, streetAddress }; Association [0..*] to Address via backlink owner; Association [0..1] to Address via backlink owner where kind=home; Association [0..*] to Address via backlink owner where zipCode like ‘76*’; Association [0..*] to Address via entity Emp2Adr; Association [0..1] to Address via entity Emp2Adr where kind=home; Association [0..*] to Address on owner=this; Association [0..*] to Address on Address.owner._id = Employee._id; Association to Address on owner=this AND kind=home;

For specifying syntax, embodiments may use a derivate of the Backus Naur Form (BNF) family of metasyntax notations used to express a context-free grammar, and which can be relied upon to make a formal description of a computer language. The basic constructs may be summarized as follows.

Construct Notation Comments definition = Definitions are written with a single equals sign, e.g. Rule = . . . extension += Extends a definition introduced before by additional rules terminal keyword Language keywords are set in bold red symbol terminal “.” Single-character language symbols are set in character double quotes alternation . . . | Pipe symbols separate alternatives, e.g. foo . . . and bar|zoo w/car grouping ( . . . ) Parenthesis group constructs, e.g. (foo|bar) with car option [ . . . ] Square brackets designate optional constructs, e.g. [optional] repetition . . . * 0+ repetitions are indicated by appended “*”, e.g. zeroOrMore* repetition . . . + 1+ repetitions are indicated by appended “+”, e.g. oneOrMore+ comment -- . . . Comments start with a double-dash, e.g. -- this is a comment

Syntax for SQL extended to include entities and associations as described herein, may be described as follows.

AssignedType += | AssociationType AssociationType = Association [ cardinality ] ( to targetEntity ) [  managedJoin | unmanagedJoin ] cardinality = “[” [( maxs |* ) “,” ] [ min .. ] ( max|* ) “]” | “[ ]” targetEntity = QualifiedName managedJoin = ( forwardLink | backwardLink | mediatedLink ) [ where  filterClause ] forwardLink = “{” foreignKeys “}” backwardLink = via backlink reverseKeys mediatedLink = via entity entityName foreignKeys = targetKeyElement [ AS alias ] [ “,” foreignKeys ] reverseKeys = targetKeyElement [ “,” reverseKeys ] targetKeyElement = elementName ( ″,″ elementName )* unmanagedJoin = on filterClause

From DDL perspective, association is a new primitive type that is specified with the type name “Association”, followed by several parameter clauses to specify requisite metadata. These parameter clauses are as follows.

Cardinality allows specifying the relationship's cardinality in the form of [min . . . max], with max=* denoting infinity and “[ ]” as a shorthand for [0 . . . *]. As a default, if omitted [0 . . . 1] is used as the default cardinality. An example is:

Association[ ] to Address via backlink owner;

To targetEntity specifies the association's target entity. A qualified name is expected, referring to another entity (incl. views). Specifying the target is mandatory—there is no default.

{foreignKeys} allows specifying a combination of alternative key elements in the target entity, to be used to establish the foreign key relationship. Where a key element is in a substructure on the target side, an alias name is to be specified. Further details are provided below regarding associations represented as foreign key relationships.

If omitted, the target entity's designated primary key elements are used. The following are examples:

Association to Address { zipCode, streetAddress }; Association to Address { some.nested.key AS snk };

Another parameter clause is VIA backlink: reverseKeys. For 1:m associations, it is mandatory to specify target elements, which are expected to be a key combination matching the source's primary keys or an association referring to the source entity. An example is:

Association to Address via backlink owner;

Another parameter clause is VIA entity: entityName. For m:m associations, it is mandatory to specify a link table's entity name. That name can either refer to a defined entity or a new entity will be created as follows:

entity <entityName> { <nameOfSourceEntity> : Association to <SourceEntity>; <nameOfTargetEntity> : Association to <TargetEntity>; }

If the data model contains an explicit definition of the link table entity, that entity must adhere to the template shown above. It can, in addition, add other elements. An example is given below:

Association to Address via entity Employee2Address; entity Employee2Address { employee : Association to Employee; address : Association to Address; }

The WHERE filterClause allows specifying additional filter conditions that are to be combined with the join conditions. This can be especially relevant in combination with VIA backlink or entity clauses. Depending on the filterCondition this can reduce a base :m relationship to one with a :1 cardinality. An example is given below:

Association to Address[0 . . . 1] via backlink owner where kind=home;

The ON filterClause allows fully specifying an arbitrary JOIN condition, which can be any standard SQL filter expression. Using this option results in the respective association being user-managed. That is, no foreign key elements/fields are created automatically. The developer is expected to explicitly manage the foreign key elements, including filling them with appropriate foreign key values in write scenarios. An example is given below:

Association to Address on owner=this;

Element names showing up in VIA, WHERE, and ON clauses, are resolved within the scope of the target entity's type structure. Siblings can be referred to by prefixing an element with a “.”. Elements from the scope above can be referred to by prefixing an element with “.”, etc.

In addition, the outer entity's top-level scope can be referred through the pseudo variable “this”, described further below in connection with Pseudo Variables in Queries (QL).

According to embodiments, associations are represented as foreign key relationships. In the relational model, associations are mapped to foreign key relationships. The foreign key elements are usually created automatically as described in the following sections. In particular, an element with association type is represented as a nested structure type containing foreign key elements corresponding to the target entity's primary key elements—i.e., having the same names and types. The following are examples of definitions which may be given:

entity Employee { ... address1 : Association to Address; address2 : Association to Address { zipCode, streetAddress }; addresses : Association to Address[0..*] via backlink owner; }

In this example, the association elements would implicitly be defined with a nested structure type containing foreign key elements in the :1 cases (plus additional metadata about the association) as follows:

entity Employee { ... address1 { _ID : type of Address._ID; } address2 { zipCode : type of Address.zipCode; streetAddress : type of Address.streetAddress; } addresses { /* none at all since :m */ } }

Following the rules for mapping structured types to the relational model as specified above, the underlying table would be created:

CREATE TABLE Employee ( ... “address1._ID” Integer, “address2.zipCode” String(...), “address2.streedAddress” String (...) )

Rules for representing associations in the persistence model may apply, as indicated in the table below:

If . . . is for to-one cases, e.g. [0 . . . 1] for to-many cases specified <no join Nested foreign key elements are not allowed clause> created for target's primary key elements. {foreignKeys} Nested foreign key elements are created for the elements specified in foreignKeys. VIA backlink No nested foreign keys are created; reverseKeys instead the reverseKeys are expected to link back from target to source. VIA entity No nested foreign keys are created; entityName instead the link table named entityName is created/used as described above. ON No nested foreign key elements are joinCondition created; managing the foreign key relationship is completely up to the developer.

Consistent with the approach in SQL, no plausibility checks are enforced (e.g. checking whether target key elements specified in {foreignKeys} fulfill the uniqueness requirements). Also, no implicit referential integrity checks are enforced at runtime.

According to embodiments, associations may be in custom-defined types. As associations are special types, they can principally be defined not only for elements in entity definitions, but in type definitions in general. For example, the following definition of the association Amount.currency is valid DDL content:

entity Currency { // List of pre-defined Currencies key code : String(3); description : String(33); } type Amount { value : Decimal(10,2); currency : Association to Currency; }

An actual relationship between entities is established when using the type Amount for an element within an entity definition, as shown in the following:

entity Employee { salary : Amount; address : Association to Address; }

The code shown above essentially indicates that the entity Employee has two associations—one association is to Address and another association is to Currency within its salary element.

Associations in custom-defined types may only be supported for a simple “to-one” relationship with a foreign key on the source side. That is, associations with via backlink or via entity clauses may not be supported for elements in custom-defined types.

Associations in Query Language (QL) are discussed below.

Querying Associations with :m Cardinality

Resolving associations or compositions with 1:m cardinality using path expressions or nested projection clauses with the flattening operator “.” in place results in flat result sets with duplicate entries for the 1: side, which is in line with standard SQL JOINs and the relational model.

As examples, in the following queries, “addresses” refers to an association with “to-many” cardinality [0 . . . *]:

SELECT name, addresses.city FROM Employee; SELECT name, addresses.{ zipCode, city } FROM Employee;

The result sets for the example queries above, are shown below, each with the same value for name repeated/duplicated for each found entry on the :m Address side:

<Result Set 1> { name, city } <Result Set 2> { name, zipCode, city }

Embodiments also allow the return of ‘Deep’ Result Sets. Specifically, in addition to the standard flattening behavior, the introduction of nested projection clauses and structured result sets principally allows expression of ‘deep’ queries along :m associations. These deep queries return ‘real deep’ result sets having the 1: sides elements on a top level, with nested tables/sets for the :m sides.

For example, the deep query:

SELECT name, addresses {zipCode, city} FROM Employee;

would be expected to return a result set with a nested collection as shown below:

<Result Set> { name, addresses : <collection of> Address { zipCode, city } }

Such deep querying may provide certain benefits. One possible benefit is to allow retrieving larger structures through a single query.

Currently, in the absence of deep querying, such larger structures may frequently be obtained in a brute-force approach, through 1+n queries with n being the number of records returned by a 1: side query. This is detrimental to performance, particularly if such a query spans several levels of to-many associations.

While the other extensions can be realized by translating to standard SQL queries, this one requires adding special support deep within the query engine. The absence of such support may preclude using to-many associations in the non-flattened way. This is discussed further below in the associations of FROM clauses, regarding how association trees can be traversed.

Associations in WHERE Clauses

Associations can arise not only in projection clauses but also in filter conditions in WHERE clauses. Respective comparison operators may be enhanced to support associations, as depicted in the following examples:

1. SELECT ... from Emloyee WHERE orgunit={ _id: ‘4711’ }; 2. SELECT ... from Emloyee WHERE homeAddress={ zipCode: ‘76149’, streetAddress: ‘Vermontring 2’ }; 3. SELECT ... from Emloyee WHERE orgunit=‘4711’; 4. SELECT ... from Emloyee WHERE homeAddress.city like ‘Wall%’; 5. SELECT ... from Emloyee WHERE homeAddress.city IN ( ‘Walldorf’, ...); 6. SELECT ... from Emloyee WHERE address IS NULL; 7. SELECT ... from Emloyee WHERE address[kind=home].city = ^(‘)Walldorf’; 8. SELECT ... from Emloyee WHERE homeAddress = addresses[kind=home];

Several issues arising within the examples immediately above, may be worthy of note. In connection with:

ad 1,2: A record literal can be passed to a comparison with an association, with elements that match the combination of the foreign keys. ad 3: Support for Association type in QL includes automatic coercions of typed scalars or string representations thereof to single-key associations. ad 4: One can also refer to the individual key values using standard path expressions. ad 5ff: Other SQL comparison operators can be used, such as LIKE, IN, IS NULL, . . . . ad 8: It can be combined with XPath-like filter expressions. ad 9: It can be combined with compare associations, provided they are assignable.

The above provides just a few examples to give the idea. In general, every condition that is possible with standard SQL expressions shall be possible to do with associations as well, including sub queries with exists and not exists, etc.

Associations in FROM Clauses

Embodiments may also allow associations in FROM clauses. Specifically, host languages may provide support for representing associations as typed variables or elements. This is described below in connection with association types in host languages.

Accordingly, one can traverse along associations, as shown in the following examples (in some pseudo language):

var daniel = SELECT name, homeAddress FROM Employee WHERE name=’Daniel’; // ... and somewhat later, maybe at some other place in an application... var addresses = SELECT * FROM Address WHERE this=daniel.homeAddress;

The expression this=<an association> can be used. The comparison this=<an association> can be retrieve an entity by a given association. The pseudo variable this is always an alias for the entity given in the FROM clause. So the statement above actually resolves to:

SELECT * FROM Address this WHERE this=daniel.homeAddress;

The comparison this=<an association> compares a queried entity with a given association—the association must be of type association to <queried entity> [0 . . . 1]. This expands to a WHERE clause corresponding to the ON condition resolved from the association. In this case it would actually resolve to:

SELECT * FROM Address this WHERE this.zipCode = daniel.homeAddress.zipCode AND this.streetAddress = daniel.homeAddress.streetAddress AND this.type = ‘home’;

Embodiments may also allow the use of SELECT from association. Specifically, association-traversal code patterns like the one below are frequently seen:

SELECT * from Address WHERE this=daniel.homeAddress;

An association in general, and a programming language variable with association type support in particular, carries all information about a target record—essentially providing information as to which entity goes with which key. Thus, equivalent to the query above, embodiments allow the shorthand below for traversing associations:

SELECT * from daniel.homeAddress;

In general, a query statement of the form SELECT . . . from <someAssociation> expands to:

SELECT ... from <someAssociation>.<targetEntity> WHERE this=<someAssociation>;

Here, <targetEntity> signifies the metadata associated with the association corresponding to the target entity specified in the association's declaration using the ON targetEntity clause.

JOINs Declare Ad-Hoc Associations

Embodiments allow JOINs to declare ad-hoc associations. In the case of a missing association, the standard JOIN <target> ON <join condition> clauses as introduced in SQL-92 are still supported, which align with the extensions introduced above, as they naturally introduce associations in an ad-hoc fashion.

For example, in the data model given above, the entity Employee has an association homeAddress, but is lacking a similar association for businessAddress, which can be compensated for using a standard JOIN clause as follows:

SELECT FROM Employee e ASSOCIATION TO Employee2Address e2a ON e2a.employee = e ASSOCIATION TO Address businessAddress ON _id = e2a.address._id AND kind=business { ID, name, businessAddress { streetAddress, zipCode, city } }

The expression may follow the syntax below:

JoinClause += | JOIN targetEntity [[AS] Identifier ] JoinConditionClauses Other syntax is as discussed above in connection with associations in DDL.

JOIN clauses fit easily into the extensions in DDL and QL. JOIN clauses can be interpreted as an ad-hoc definition of missing associations.

In the example immediately above, the association businessAddress is added. This result is recognized if the projection clause of the example above is compared to that of the query applied to the domain model if the association were in place (below):

SELECT FROM Employee { ID, name, businessAddress { streetAddress, zipCode, city } }

Embodiments also allow the use of simplified JOIN clauses. In particular, following the observation that JOINs essentially declare ad-hoc associations, embodiments allow JOINs to be declared using the same clauses that are used to declare associations in DDL. Given this, the above example can be written more easily as follows:

SELECT FROM Employee e ASSOCIATION TO Address businessAddress VIA entity Employee2Address WHERE kind=business { ID, name, businessAddress { streetAddress, zipCode, city } }

FIG. 6 illustrates hardware of a special purpose computing machine configured to implement transient fields according to an embodiment. In particular, computer system 600 comprises a processor 602 that is in electronic communication with a non-transitory computer-readable storage medium 603. This computer-readable storage medium has stored thereon code 604 corresponding to a query engine. Code 605 corresponds to a database engine. Code may be configured to reference data stored in a database of a non-transitory computer-readable storage medium, for example as may be present locally or in a remote database server. Software servers together may form a cluster or logical network of computer systems programmed with software programs that communicate with each other and work together in order to process requests.

An example computer system 710 is illustrated in FIG. 7. Computer system 710 includes a bus 705 or other communication mechanism for communicating information, and a processor 701 coupled with bus 705 for processing information. Computer system 710 also includes a memory 702 coupled to bus 705 for storing information and instructions to be executed by processor 701, including information and instructions for performing the techniques described above, for example. This memory may also be used for storing variables or other intermediate information during execution of instructions to be executed by processor 701. Possible implementations of this memory may be, but are not limited to, random access memory (RAM), read only memory (ROM), or both. A storage device 703 is also provided for storing information and instructions. Common forms of storage devices include, for example, a hard drive, a magnetic disk, an optical disk, a CD-ROM, a DVD, a flash memory, a USB memory card, or any other medium from which a computer can read. Storage device 703 may include source code, binary code, or software files for performing the techniques above, for example. Storage device and memory are both examples of computer readable mediums.

Computer system 710 may be coupled via bus 705 to a display 712, such as a cathode ray tube (CRT) or liquid crystal display (LCD), for displaying information to a computer user. An input device 711 such as a keyboard and/or mouse is coupled to bus 705 for communicating information and command selections from the user to processor 701. The combination of these components allows the user to communicate with the system. In some systems, bus 705 may be divided into multiple specialized buses.

Computer system 710 also includes a network interface 704 coupled with bus 705. Network interface 704 may provide two-way data communication between computer system 710 and the local network 720. The network interface 704 may be a digital subscriber line (DSL) or a modem to provide data communication connection over a telephone line, for example. Another example of the network interface is a local area network (LAN) card to provide a data communication connection to a compatible LAN. Wireless links are another example. In any such implementation, network interface 704 sends and receives electrical, electromagnetic, or optical signals that carry digital data streams representing various types of information.

Computer system 710 can send and receive information, including messages or other interface actions, through the network interface 704 across a local network 720, an Intranet, or the Internet 730. For a local network, computer system (710 may communicate with a plurality of other computer machines, such as server 715. Accordingly, computer system 710 and server computer systems represented by server 715 may form a cloud computing network, which may be programmed with processes described herein. In the Internet example, software components or services may reside on multiple different computer systems 710 or servers 731-735 across the network. The processes described above may be implemented on one or more servers, for example. A server 731 may transmit actions or messages from one component, through Internet 730, local network 720, and network interface 704 to a component on computer system 710. The software components and processes described above may be implemented on any computer system and send and/or receive information across a network, for example.

The above description illustrates various embodiments of the present invention along with examples of how aspects of the present invention may be implemented. The above examples and embodiments should not be deemed to be the only embodiments, and are presented to illustrate the flexibility and advantages of the present invention as defined by the following claims. Based on the above disclosure and the following claims, other arrangements, embodiments, implementations and equivalents will be evident to those skilled in the art and may be employed without departing from the spirit and scope of the invention as defined by the claims. 

What is claimed is:
 1. A computer-implemented method comprising: providing a database comprising data of a first type and data of a second type; causing a database engine to receive from a query engine, a query in a database language defining a transient field; causing the database engine to communicate with the database to generate a query result from the query, wherein the database engine fills in a value for the transient field derived at least in part from the data of the second type; and causing the database engine to communicate the query result to the query engine for display to a user.
 2. A computer-implemented method as in claim 1 wherein the database language comprises SQL.
 3. A computer-implemented method as in claim 1 wherein the transient field is defined by an identifier in the database language.
 4. A computer-implemented method as in claim 1 wherein the database language is extended to include an entity, and the transient field is defined by the entity.
 5. A computer-implemented method as in claim 1 wherein the value is not materialized in a persistent field in the database.
 6. A computer-implemented method as in claim 1 wherein the value is materialized in a persistent field of the database based upon a heuristic or a hint.
 7. A non-transitory computer readable storage medium embodying a computer program for performing a method, said method comprising: providing a database comprising data of a first type and data of a second type; causing a database engine to receive from a query engine, a query in a database language defining a transient field; causing the database engine to communicate with the database to generate a query result from the query, wherein the database engine fills in a value for the transient field derived at least in part from the data of the second type; and causing the database engine to communicate the query result to the query engine for display to a user.
 8. A non-transitory computer readable storage medium as in claim 7 wherein the database language comprises SQL.
 9. A non-transitory computer readable storage medium as in claim 7 wherein the transient field is defined by an identifier in the database language.
 10. A non-transitory computer readable storage medium as in claim 7 wherein the database language is extended to include an entity, and the transient field is defined by the entity.
 11. A non-transitory computer readable storage medium as in claim 7 wherein the value is not materialized in a persistent field in the database.
 12. A non-transitory computer readable storage medium as in claim 7 wherein the value is materialized in a persistent field of the database based upon a heuristic or a hint.
 13. A computer system comprising: one or more processors; a software program, executable on said computer system, the software program configured to: provide a database comprising data of a first type and data of a second type; cause a database engine to receive from a query engine, a query in a database language defining a transient field; cause the database engine to communicate with the database to generate a query result from the query, wherein the database engine fills in a value for the transient field derived at least in part from the data of the second type; and cause the database engine to communicate the query result to the query engine for display to a user.
 14. A computer system as in claim 13 wherein the database language comprises SQL.
 15. A computer system as in claim 13 wherein the transient field is defined by an identifier in the database language.
 16. A computer system as in claim 13 wherein the database language is extended to include an entity, and the transient field is defined by the entity.
 17. A computer system as in claim 13 wherein the value is not materialized in a persistent field in the database.
 18. A computer system as in claim 13 wherein the value is materialized in a persistent field of the database based upon a heuristic or a hint. 